Skip to content

Prepare the ten-page curator quality retest - #296

Merged
holden merged 2 commits into
mainfrom
codex/275-quality-preflight
Nov 1, 2026
Merged

holden merged 2 commits into
mainfrom
codex/275-quality-preflight

Conversation

@holden

@holden holden commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

#275 still has a quality NO-GO despite a working publication pipeline. Freeze a prospective scorecard with explicit uncertainty/failure denominators; report historical eligible pools, bounded packets, selections and publication/current eligibility from existing records; clarify in immutable instruction v6 that an eligible supplied sense_definition may lead without a dictionary content entry.

Preserves v1–v5 instruction hashes, Bierce priority, output-v4 shape, exact citations, fixed ten pilots, existing budgets and justified abstention. Adds only an admission migration for later approved rollout; no new tables/source store/agent framework. The report starts no server, peer, queues/plugins or model and uses read-only connections/transaction.

Validation at final head c533c76:

  • Full mix precommit: 19 doctests, 3,311 tests, zero failures, exit0 (489.6s), private partition _q275 on5434.
  • 48 focused checks pass, including sense-only exact publication, payload-free reporting and migration up/down/up/rollback refusal.
  • Independent exact-head review: no required changes; final passing test record supplied for confirmation.
  • Actual read-only v2 coverage report is documented in docs/curation/coverage-2026-10-31.md. Snapshot uses deployed retention policy, explicitly replacing shortened test windows; discarded initial snapshot is disclosed. Six withheld choices match the audit. Historical counts are unchanged.
  • Before/after live checks:35 attempts, editor off,39 migrations, expected cluster7693849764459364596. Main remains49dbf87.

The first full test run lacked PostgreSQL utilities on PATH and failed; it is not counted as passing. The complete corrected-environment rerun above passed. No code workaround was introduced for those failures.

Scorecard SHA256:156ee53110b7c8ecad98291a8087c91b831e2a07c1c47f19aca4e269ee7cd1a9. V6 instruction SHA256:e037baf1f599d321ea35ba740bb0d5deaf3bbfb880ec47ce6c33bdfb96e67212.

No live migration, model registration/start/call, activation, deployment, enrollment or panel. This is preparation, not a model-quality pass or authorization for the multi-day experiment. #275/#271 remain open.

Summary by CodeRabbit

  • New Features

    • Whole-entry curation can now use an eligible sense definition as the lead when it clearly defines an entry, including when no content candidate is available.
    • Added a read-only coverage report with attempt summaries, selection and omission counts, and current publication details.
  • Documentation

    • Added a dated trial status and evaluation scorecard. The scorecard is not run and does not authorize model calls, activation, or deployment.
    • Documented report boundaries, evidence-handling rules, and rollback constraints.

@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

This PR adds Voltaire whole-entry instruction v6, including support for an eligible sense definition as the lead. It adds a bounded, read-only coverage report and documents the retest scorecard, reporting limits, and authorization boundaries.

Changes: Curation retest preparation

Layer / File(s) Summary
Whole-entry v6 instruction and admission
lib/devils_dictionary/curation/runtime/packet.ex, lib/devils_dictionary/curation/runtime/contract.ex, lib/devils_dictionary/curation/runtime/section_output.ex, lib/devils_dictionary/curation/whole_entry.ex, lib/devils_dictionary/curation/agents/persona.ex, priv/curation/agents/personas/voltaire/v7/profile.json, lib/mix/tasks/dd.curation.editor.ex, priv/repo/migrations/20261101000313_admit_whole_entry_sense_lead_instruction.exs, test/devils_dictionary/curation/runtime/packet_test.exs, test/devils_dictionary/curation/whole_entry_test.exs, test/devils_dictionary/installation/whole_entry_compatibility_test.exs
Instruction v6 can select an eligible sense_definition as the lead when no content candidate is supplied. Runtime and Voltaire profile handling accept v6. The migration admits v6 configurations and refuses rollback when v6 history exists. Tests cover instruction compatibility, publication, and rollback behavior.
Read-only coverage report
lib/devils_dictionary/curation/coverage.ex, lib/mix/tasks/dd.curation.coverage.ex, test/devils_dictionary/curation/coverage_test.exs, docs/curation/coverage-2026-10-31.md
The report bounds its page, attempt IDs, and result count. It summarizes matching attempts and current publication data. The Mix task reads in a repeatable-read, read-only transaction and prints JSON. Tests and the historical snapshot document report fields, limits, and caveats.
Evaluation scope and authorization boundaries
docs/curation/agents-mvp.md, docs/curation/scorecard-v2-2026-10-31.md
The documents record the NO-GO preflight status, define the ten-page scorecard and quality thresholds, and specify operational measures and evidence limits. They state that model calls and v6 activation require separate authorization.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MixTask as Mix.Tasks.Dd.Curation.Coverage
  participant Coverage as DevilsDictionary.Curation.Coverage
  participant Repo
  participant Receipt as Receipt.read/2
  MixTask->>Coverage: read pilot page coverage
  Coverage->>Repo: retrieve matching attempts and publication data
  Coverage->>Receipt: read current publication receipt
  Coverage-->>MixTask: return bounded report
  MixTask-->>MixTask: print formatted JSON
Loading

Possibly related issues

  • Issue 275: The PR prepares the documented ten-page retest and adds whole-entry v6, while retaining the NO-GO status and separate authorization requirement.
  • Issue 271: Instruction v6 makes an eligible supplied sense definition selectable as a lead.
  • Issue 253: Instruction v6 addresses the open question of whether a sense definition can lead without a content entry.

Possibly related PRs

  • razrfly/dictionary#278: This PR extends the single_editor_v1 publication path with Voltaire v6 admission support.
  • razrfly/dictionary#276: This PR adds the v7 Voltaire manifest and supports its v6 whole-entry persona version.
  • razrfly/dictionary#279: This PR continues the whole-entry curation path with v6 and the sense-definition lead rule.
Merge Risk: Low · up to c533c

The coverage report cannot run for a single slug using the documented command. Fix that invocation before relying on it; reporting all pilots remains available.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@holden
holden marked this pull request as ready for review November 1, 2026 00:41

holden commented Nov 1, 2026

Copy link
Copy Markdown
Member Author

Independent final review: CODE GO — REQUIRED changes none, exact head c533c76 / tree f6c66f645250feb322cdb339cb657fbfdb389371. Reviewer independently verified the passing final precommit log:19 doctests,3,311 tests,zero failures,489.6s. All four initial review findings are resolved. Corrected coverage evidence matches its report artifact. This is code GO for merge review, not editorial-quality acceptance or permission to migrate/deploy/start/activate/expand.

Retained private test database: devils_dictionary_test_q275 on5434, external drive. No live migration applied; live main49dbf87,editor/model off,35 attempts,39 migrations.

@holden
holden merged commit 9a17333 into main Nov 1, 2026
1 of 2 checks passed
@holden
holden deleted the codex/275-quality-preflight branch November 7, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant